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(54) Multimedia information coding apparatus, coding method of multimedia information, and 
recording media storing data coded by the same method 

(57) A multimedia title including sources of plural 
data-types such as a motion picture, still picture and text 
data, presentation methods of respective sources, and 
the data-types is edited in every frame, so that a video 
signal is created. Based on a scenario, the video signal 
is coded adaptively to respective data-types of each 
source. As a result, calculation volume is reduced and 
picture quality is improved. 
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Description 

Field of the Invention 

[0001 ] The present invention relates to a technique of 
editing a multimedia title which comprises various 
sources such as motion pictures, still pictures, text data, 
background still image, graphics, and a scenario 
describing the source list, presentation method and 
characteristics of the sources into video signals. It also 
relates to a technique of coding a type of sequential 
video signals into coded video signals such as MPEG 
responsive to a type of data. 

[0002] In this specification, the technical terms used in 
the above paragraph are defined as follows: 

"source" : a displayed unit of respective types of 
information such as motion pictures, still pictures, 
text data, background still image, graphics, and the 
like 

"scenario" : a method of displaying a source, e.g. a 
timing and a location in a screen of a source to be 
output 

"multimedia title" : data comprising sources and 
their scenarios 

"data type" : characteristics of a source, such as a 
type of information, picture quality, application, and 
process provided 

Background of the Invention 

[0003] In recent years awareness of multimedia has 
increased in the market, and video-on-demand-systems 
have drawn consumers' attention because this system 
can provide users with moving pictures on a television 
set or a personal computer anytime when the users 
want to watch them. 

[0004] In the video-on-demand-system, a video com- 
prising various data such as motion pictures, still pic- 
tures, text data and the like is presented as a multimedia 
title so that an efficient presentation can be achieved. 
Therefore, the market has demanded a multimedia-on- 
demand-system that delivers multimedia titles respon- 
sive to requests from respective terminals, which then 
reproduce the titles. 

[0005] The following two methods are available for 
reproducing the multimedia title. 

(1) The reproduction terminal relevantly repro- 
duces, decodes and edits respective sources fol- 
lowing their scenarios, and then displays them at a 
specified time in a specified region so that a plural- 
ity of videos can be edited on a screen. 

(2) The terminal reproduces respective sources 
based on information from their scenarios by using 
reproducing methods relevant to each source, and 
edits them on a screen. 



[0006] According to the methods discussed above, the 
terminal must display respective sources in accordance 
with their characteristics e.g. text data is developed into 
fonts. In the case of multimedia title including motion 

5 pictures, the terminal must have versatile functions such 
as displaying motion pictures, still pictures and text 
data. As such, reproduction of the multimedia title 
demands a lot of functions of the terminal, thus reducing 
processes in the terminal is needed. 

10 [0007] A technique for reducing a number of functions 
in the terminal is disclosed in the Japanese Patent 
Application Non-examined Publication No. H06- 
301 495. It teaches that if the terminal has no character- 
font-information, the data to be input to the terminal can 

75 be equipped with some character fonts responding to 
characters used in a content of a presentation so that 
the characters can be reproduced. However, this 
method does not contribute to reduce a lot of functions 
in the terminal. 

20 [0008] Another method is that a combination of e.g. a 
scan converter with a hard encoder converts the multi- 
media title into coded video signals. Most of the scan 
converters only accept sources converted into analog- 
signals, and this conversion deteriorates picture quality. 

25 The hard encoder does not provide an optimal coding to 
the sources such as still pictures and text data which 
can predict the next frame, but provide the same coding 
as other sources such as motion pictures. Unnecessary 
coding is thus provided, which lowers picture quality 

30 and demands a lot of calculation. The hard encoder also 
provides the same weighing as other sources — natural 
images including motion and still pictures — to text data, 
graphics or monotonous artificial images. These artifi- 
cial images are painted with less numbers of colors and 

35 can be used in background. Then the weighted data are 
quantized, and thus picture quality is towered. 

Summary of the Invention 

40 [0009] The present invention addresses the problems 
discussed above and aims to achieve the following tar- 
gets. 

(1) Realize a low cost reproduction terminal by edit- 
45 ing a multimedia title into a type of video signals so 

that a necessary process for the terminal is only a 
reproduction of motion pictures. 

(2) Code the edited video signals so that the follow- 
ing objectives can be realized: 2-1 . Improving pic- 

so ture quality, 2-2. Reducing a number of 
calculations, and 2-3. Improving a coding rate. 

[001 0] In order to achieve the goals discussed above, 
the present invention edits a multimedia title into a type 
55 of video signals such as motion pictures. The multime- 
dia title comprises various sources including motion pic- 
tures, stilt pictures, text data, background still pictures, 
and the like as well as scenarios including the source 
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list, methods of presentation and characteristics of the 
sources. 

[0011} The present invention then codes the edited 
video signals into coded video signals such as MPEG. 
When coding, the present invention utilizes the scenar- 5 
ios, and thereby assigning relevant coding methods to 
respective sources based on a ratio of region areas 
where respective data types appear. As a result, 
respective sources can be coded in accordance with 
their data types. w 
[0012] In other words, a multimedia title comprising 
various kinds of sources and their scenarios is con- 
verted into a type of coded video and audio signals fol- 
lowing some format such as MPEG. The reproduction 
terminal thus can be equipped with only a function is 
decoding and reproducing the type of coded video and 
audio signals. 

[0013] When converting the edited video signals into 
the coded video signals such as MPEG, the scenarios 
contribute to code the signals in a suitable manner for so 
the characteristics of the multimedia title such as kinds 
of sources, ratios of respective sources making up the 
title. In this case, the scenarios also contribute to code 
the signals relevant to data types of respective sources 
responsive to the regions where different data types are 25 
displayed. As a result, coding rate is increased, a 
number of calculations are decreased, and picture qual- 
ity is improved because the coding method accommo- 
dates respective data types such as text data. 

30 

Brief Description of the Drawings 



[0014] 



Fig. 1 is a block diagram of an information editor for 35 
multimedia in accordance with a first exemplary 
embodiment of the present invention. 
Fig. 2 is a flowchart illustrating a process-flow per- 
formed by a multimedia information coding appara- 
tus. 40 
Fig. 3 is a schematic diagram depicting an example 
of information making up a frame. 
Fig. 4 is a flowchart depicting a frame analyzing 
process which creates the information making up 
the frame.. 45 
Fig. 5 illustrates components of a frame. 
Fig. 6 is a flowchart illustrating a process which cre- 
ates frame components of a motion picture source. 
Fig. 7 is a flowchart illustrating a still-picture-decod- 
ing-process which creates frame components out so 
of a source of the still picture. 
Fig. 8 is a flowchart illustrating a font-decoding- 
process which creates frame components out of a 
source of text data. 

Fig. 9 is a flowchart illustrating a process of frame ss 
editing. 

Fig. 10 is a block diagram of an apparatus which 
analyzes a scenario and thereby producing coded 



video signals in accordance with the data type of 
the scenario. 

Fig. 1 1 depicts an image of a scenario. 
Fig. 12 details the image of the scenario. 
Fig. 1 3 is a block diagram of an encoder adaptive to 
a data type used in a second exemplary embodi- 
ment. 

Fig. 14 depicts a rough image of one frame of a 
multimedia title. 

Fig. 15 depicts an image of helper-file-information 
for data type adaptive coding. 
Fig. 16 depicts an image of control file of quantizer. 
Rg. 17 depicts an image of qunatization character- 
istics data adaptive to a data type. 
Rg. 18 is a flowchart of a scenario analyzer. 
Rg. 19 is a process flowchart of an encoder adap- 
tive to a data type, the encoder controls quantizer. 
Fig. 20 is a flowchart illustrating that a creator of 
data-type-adaptive-helper-file creates the file by 
analyzing a scenario. 

Rg. 21 is a flowchart illustrating a process of pro- 
ducing a quantizer control file by data-type-adap- 
tive-encoder-control-means using a helper-file of 
an encoder adaptive to a data type. 
Rg. 22 is a block diagram of an encoder adaptive to 
a data type used in accordance with a third exem- 
plary embodiment. 

Rg. 23 is a flowchart illustrating a process of the 
encoder adaptive to a data type in accordance with 
the third exemplary embodiment. 
Rg. 24 is a flowchart illustrating a process of pro- 
ducing an intra-flag-information-file by data-type- 
encoder-control-means in accordance with the third 
exemplary embodiment using a helper-file of an 
encoder adaptive to a data type. 
Rg. 25 is an image of an intra-flag-information-file. 
Rg. 26 is a block diagram of an encoder adaptive to 
a data type in accordance with the fourth exemplary 
embodiment. 

Fig. 27 is a flowchart illustrating a process of the 
encoder adaptive to a data type in accordance with 
the fourth exemplary embodiment. 
Fig. 28 is a flowchart illustrating a process of pro- 
ducing an escape-information -file by data-type- 
encoder-control -means using a helper file of an 
encoder adaptive to a data type. 
Rg. 29 is an image of the escape information file. 
Fig. 30 is an image of a quantizer control file in 
accordance with a fifth exemplary embodiment. 
Rg. 31 is an image of matrix information of quanti- 
zation adaptive to data type. 
Rg. 32 is a flowchart illustrating a process of the 
encoder adaptive to a data type in accordance with 
the fifth exemplary embodiment. 
Rg. 33 is a flowchart illustrating a process of pro- 
ducing a quantizer-control-file by data-type-adap- 
tive-encoder-control-means using a helper file of an 
encoder adaptive to a data type in accordance with 
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the fifth exemplary embodiment. 
Fig. 34 is a block diagram of the encoder adaptive 
to a data type in accordance with the sixth exem- 
plary embodiment. 

Fig. 35 is an image illustrating one macro block in s 
one frame. 

Fig. 36 is an image of an enlarged macro block 
shown in Fig. 35. 

Fig. 37 is a block diagram of an encoder adaptive to 
a data type in accordance with the eighth exem- w 
plary embodiment. 

Fig. 38 is an image of a region where regular 
motion estimation can be searchable in one frame. 
Fig. 39 is an image of a narrowed region where 
motion estimation can be searchable in one frame, is 
Fig. 40 is an image of a deviated region where 
motion estimation can be searchable in one frame. 
Fig. 41 is an image of encoding adaptive to a text 
data in a series of frames in accordance with a ninth 
exemplary embodiment. 20 
Fig. 42 is a block diagram of an encoder adaptive to 
a data type in accordance with the ninth exemplary 
embodiment. 

Detailed Description of the Preferred Embodiments 25 
Exemplary Embodiment 1 

[001 5] In this first exemplary embodiment, an appara- 
tus converting a multimedia title into a type of coded 30 
audio and video signals in accordance with a standard 
format such as MPEG is demonstrated. 
[001 6] As shown in Fig. 1 , motion picture memory 1 0 1 
stores a source of a motion picture, still picture memory 
102 stores a source of a still picture, text memory 103 35 
stores a source of text. Motion picture decoder 105 
decodes a motion picture and then reproduces respec- 
tive frames of audio and motion picture. Still motion 
decoder 106 decodes a source of a still picture into 
frame components, and text decoder 107 decodes a 40 
character into frame components. Frame component 
memory 108 stores frame components. The members 
discussed above and members to be described in the 
following steps create a type of coded audio and video 
signals. *s 
[0017] The process flowchart of this apparatus is 
described hereinafter with reference to Fig. 1 and Fig. 2. 

Step 11: Scenario analyzer 1001 obtains a sce- 
nario stored in scenario memory 104, then creates so 
frame make-up information based on the obtained 
scenario, and stores the information in frame make- 
up memory 110. 

Step 12: Frame editor 112 instructs motion picture 
decoder 105, still picture decoder 106 and text ss 
decoder 107 to decode respective sources and cre- 
ate necessary components, and then, frames mak- 
ing up one bit stream are created. The frames are 



stored in frame buffer 113. 
Step 13: Motion picture coding means 115 codes 
the frame data stored in frame buffer 1 13, thereby 
producing a coded video stream. 
Step 14: Audio coding means 1 14 codes audio sig- 
nals of sources stored in audio component memory 
111, thereby producing a coded audio stream. 
Step 15: Multiplexer 116 multiplexes the coded bit 
stream and coded audio stream, thereby producing 
coded audio and video signals. 

[0018] The following alternatives are available in this 
process-flow: Step 13 can be processed after Step 14, 
or both the steps can be started before Step 12 is com- 
pleted. Thus both the steps can be processed along 
with Step 12 or both the steps alternate with Step 12, 
whereby frames and audio components created are 
sequentially coded. 

[0019] The features of this apparatus are detailed 
hereinafter. The apparatus edits and codes a multime- 
dia data including various and plural sources into a type 
of coded audio and video signals. This is achieved 
through step 1 1 (scenario analyzing process) and step 
12 (frame editing process). These two steps further 
detailed hereinafter with reference to Fig. 1 and Fig. 3 
through Fig. 9. 

Scenario analyzing process: 

[0020] Information making up a frame is detailed first. 
This information is a base for frame editor 112 to edit 
frames, and describes data of sources displayed in 
respective frames of videos reproduced from the coded 
audio and video signals supplied from the apparatus. 
[0021 ] Fig. 3 illustrating an example of frame making - 
up information which describes the following content: 

(A) entire frames, (B) information of each frame, and (C) 
content of each source included in one frame 

[0022] As shown in Fig. 3, each frame information 
comprises the following elements: 

(a) a frame number indicating the position of the 
frame from the first frame; 

(b) a frame size indicating a number of pixels in ver- 
tical and horizontal lines; and 

(c) source information 

[0023] The source information comprises the following 
elements: 

(1) a data type indicating a type of data, i.e. the 
source is a motion picture, still picture or text. 

(2) a source ID in a title, indicating a value proper to 
the source; 

(3) the coordinates of a staring point (upper left cor- 
ner) of displaying the source in the frame; and 
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(4) the coordinates of an ending point (lower right 
corner). 

[0024] Fig. 4 is a flowchart illustrating an analyzing 
process of a frame, this process creates the information s 
making up the frame. 

Step 1101: Scenario analyzer 1001 reads a sce- 
nario stored in scenario memory 104. 
Step 1 102: Find a number of frames of a title. io 
Step 1103: Prepare a skeleton, which describes a 
number of frames up to the total number, in each 
frame. 

Step 1104: End the process if no source is left, 
which has been described in the scenario but not is 
reflected to the information making up the scenario. 
If such sources are left, then go to Step 1 105. 
Step 1 105: Scenario analyzer 1001 issues a proper 
source ID to the sources not reflected yet Then the 
analyzer obtains those elements from the scenario 20 
such as a data type, the coordinates of starting and 
ending points in a display region, starting and end- 
ing times (indicated with frame numbers), and a 
source file name. 

Step 1106: Scenario analyzer 1001 describes the 25 
source information discussed above in the frames 
bearing respective frame numbers within a time 
region from display start time to display end time, 
then checks whether unprocessed sources still 
remain. If unprocessed sources still remain, repeat so 
the process from Step 1 104 and onward. 

[0025] Components making up a frame are described 
hereinafter. As shown in Fig. 5, the component com- 
prises the following elements: 35 

(a) a source ID; 

(b) a number of pixels in vertical and horizontal 
lines; 

(c) luminance signal; and 40 

(d) color difference signals 

[0026] The component has an identical data structure 
and a video format regardless of types of data such as 
text, motion picture, or still picture. One kind of video for- 45 
mat is used (such as 4:22 which motion picture 
encoder 1 15 can utilize directly is used.) Thus frames 
can be edited with ease by using the frame compo- 
nents. 

[0027] As shown in Fig. 6, motion picture encoder 1 05 so 
creates the frame components out of the sources of 
motion pictures following the processes shown below. 

Step 1301: Take one frame of a motion picture 
source out of the source memory 101 into encoder 55 
105. 

Step 1302: Decode the source into frame data, and 
at the same time, decode audio when the source 



8 

includes audio, and store it into audio component 
memory 111. 

Step 1303: If the decoded frame data is different 
from an instructed video format of the frame com- 
ponents, convert it into the instructed one. 
Step 1304: If the decoded frame has a size different 
from an instructed data size, convert it to the 
instructed one. 

Step 1305: Add a source ID to the frame compo- 
nent, and store the component in frame component 
memory 108. 

Step 1306: End the process when all the frames of 
motion picture sources are processed. If some 
frames are not processed yet. repeat the process 
1301 and onward. 

[0028] As shown in Fig. 7, still picture decoder 106 
decodes a still picture following the steps shown below. 

Step 1311: Take a source of still picture out of still- 
picture-component-memory 102 into decoder 106. 
Step 1312: Decode the source into still picture data. 
Step 1313: When a video format of the decoded still 
picture data is different from that of the frame com- 
ponents, decoder 106 converts it to the video for- 
mat of the frame components. 
Step 1314: When the decoded sill picture has a dif- 
ferent size from an instructed size, convert it into 
the instructed size. 

Step 1315: Add a source ID to the frame compo- 
nent, and store the component in frame component 
memory 108. 

[0029] As shown in Fig. 8, text decoder 1 07 processes 
font decoding following the steps shown below. 

Step 1321 : Take a source of text out of text compo- 
nent memory 103 into decoder 107. 
Step 1322: Create a window having an instructed 
size for decoding text. 

Step 1323: Decode the text data on the window, 
and create bit map data. 

Step 1324: Convert a video format of the bit map 
data into a video format of the frame component. 
Step 1325: Add a source ID to the frame compo- 
nent, and store the component in frame component 
memory 108. 

[0030] Frame editor 112 instructs the productions of 
frame components discussed above. 
[0031 ] Frame editing is detailed hereinafter. As shown 
in Fig. 9. frame editor 112 edits frames following the 
steps shown below. 

Step 1201 : Take the information making up a frame 
out of frame-makeup-information-memory 110 into 
frame editor 112. 

Step 1202: Draw a background on a frame to be 
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edited. 

Step 1203: Obtain information of sources not yet 
processed in the frame to be edited, and search 
whether the component with the source ID obtained 
is stored in frame component memory 108 or not. 
Step 1204: When a necessary component is not 
stored in memory 108, go to Step 1205. 
Step 1205: Instruct one of motion picture decoder 
105, still picture decoder 106 or text decoder 107 to 
create a frame component in accordance with the 
data type described in the source information. 
Step 1206: When a necessary component is stored 
in memory 108, obtain the frame component to be 
used from frame component memory 111, and 
draw it on a position described in the source infor- 
mation of the f rame. 

Step 1207: If frame components to be edited 
remain in the frame, search the components. 
(Repeat the processes from Step 1203 and on 
ward.) 

Step 1208: If the frame components to be edited do 
not remain anymore, output frame data to frame 
buffer 113. 

Step 1209: When each frame has been edited, end 
the process. If a frame to be edited still remains, 
repeat the process from Step 1202 and onward to 
edit the remained frame. 

[0032] The apparatus in accordance with this first 
exemplary embodiment decodes source data based on 
information obtained by analyzing a scenario. The vari- 
ous kind of and plural source data as well as the sce- 
nario make up multimedia data. The apparatus thus 
converts the multimedia data into a type of coded audio 
and video signals such as MEPG 2. 
[0033] Processes once needed for decoding and dis- 
playing various kinds of source data in order to survey 
multimedia data are thus eliminated in this apparatus. 
This apparatus needs only processes of decoding and 
displaying a type of coded audio and video signals such 
as MPEG in order to survey the multimedia data. 
[0034] In this embodiment, a motion picture, still pic- 
ture, text and audio are used as sources to be respon- 
sive to a multimedia title as shown in Fig. 14, of which 
video signals comprises a moving picture, still picture, 
and text. However, these four sources are not necessar- 
ily prepared. 

Exemplary Embodiment 2 

[0035] An apparatus used in the second exemplary 
embodiment can code sources adaptively to a type of 
their data in an efficient manner based on information 
by scenario analysis. This is a function additional to the 
converting function of multimedia information into coded 
video signals, which has been discussed in the first 
exemplary embodiment. 

[0036] This embodiment demonstrates to code the 



data adaptively to the data type by controlling a quan- 
tizer based on a scenario comprising information about 
region, time, and data type. 

[0037] Region information indicates a region where 
s data appears, and the region is usually described by 

two points on X and Y coordinates. 

[0038] Time information indicates a time when data 

appears. The time has a starting and ending points and 

is usually described by a unit of frame. 
to [0039] Data type information indicates a data type of 

file which is a source of multimedia information, and it is 

described as a motion picture type, still picture type or a 

text type. 

[0040] The apparatus in accordance with the second 

is embodiment is constructed as shown in Fig. 1 0, i.e. sce- 
nario analyzer 1001 includes a data-type-adaptive- 
helper-file-creator 1002 which creates the helper-file by 
using a scenario comprising the information of region, 
time and data type. This helper file is shown in Fig. 15 

20 and is used for coding the sources. Data-type-adaptive- 
coding-section 1003 includes data-type-adaptive- 
encoder 1005 and data-type-adaptive-coding-controller 
1004. Section 1003 codes the sources adaptively to 
respective data type of the sources. 

25 [0041 ] In encoder 1 005 shewn in Fig. 1 3. input-frame- 
buffer 1 006 stores the frame data of a edited frame, and 
input-signal-identifier identifies a coding type of the 
frame data supplied whether the coding type is one of I 
frame, B frame or P frame. Orthogonal transformer 

30 1 008 provides the supplied video signals with a discrete 
cosine transformer (DCT) process. Quantizer 1009 
quantizes the input data based on a process mode 
decided by controller 1004. Inverse quantizer 1010 
inversely quantizes the data quantized. Inverse orthog- 

35 onal transformer 1011 provides the data inversely quan- 
tized with an inverse DCT process, thereby producing 
uncoded video signals. 

[0042] Frame decoder 1012 decodes a frame based 
on the data from motion-vector-buffer 1015 and refer- 

40 ence-frame-buffer 1013. Reference-frame-buffer 1013 
stores the data of a frame decoded by decoder 1012. 
Motion estimator 1014 follows an identifying by identifier 
1007 and estimates a motion based on the data from 
input-frame-buffer 1006, reference-frame-buffer 1013 

45 and motion vector-buffer 1015. Buffer 1015 stores 
motion-vector-data created when estimator 1014 esti- 
mates motions. 

[0043] Variable length encoder 1016 codes frame 
data in a variable length manner, then the frame data 

so has been qunatized by quantizer 1 009 using the motion- 
vector-data stored in buffer 1015. 
[0044] An operation of data-type-adaptive-apparatus 
thus constructed is described hereinafter with reference 
to Fig. 11, Fig. 12, Fig. 15. Fig. 16, and Fig. 18 through 

55 Fig. 21 . 

[0045] A flowchart of the process by the apparatus 
used in the second embodiment includes the process 
described in the first embodiment and a process of cod- 
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ing sources adaptively to their data types. 
[0046] This process of coding the sources adaptively 
to their data type can be realized by combining a proc- 
ess by scenario analyzer 1 001 and a process by coding 
section 1 003. The process flow is described hereinafter, s 
[0047] Helper-file-creator 1002 as shown in Fig. 18 
analyzes a scenario following the process shown below. 



[0048] A process by data-adaptive-coding-section 
1003 is described in Fig. 19. 

is 

Step 103: Section 1003 reads the frame data cre- 
ated in frame editor 112 and stored in frame buffer 
113. 

Step 104: Store the frame data read by section 
1003 into input-frame-buffer 1006. 20 
Step 105: Coding controller 1004 reads the helper- 
file created in the scenario analyzer. 
Step 106: Coding controller 1004 creates quantizer 
controlling file shown in Fig. 16 based on the helper 
file shown in Fig. 15. 25 
Step 107: Controller 1004 transmits the quantizer 
controlling file to data-type-adaptive-quantizer 
1009. 

Step 108: Input identifier 1007 identifies a frame to 
be created. 30 
Step 109: If a frame to be created is a predictive 
frame (P-frame) or bi-directional predictive frame 
(B-frame), motion estimator 1014 estimates a 
motion by differentiates between the frame data 
read by identifier 1007 and the frame data stored in 35 
buffer 1013. 

Step 110: Motion estimator 1014 stores motion- 
vector-data into motion-vector-buffer 1015. 
Step 111: Orthogonal transformer 1008 transforms 
the data from identifier 1007 if the frame to be ere- 40 
ated is an intra-frame (l-frame.) Motion estimator 
1014 transforms the data estimating the motions if 
the frame to be created is a predictive frame (P- 
frame) or a bi-directional predictive frame (B- 
frame.) 45 
Step 112: Quantizer 1009 has received the control- 
ling file shown in Fig. 1 6 and created by coding con- 
troller 1004. Quantizer 1 009 determines quantizing 
characteristics responsive to a macro block ID, and 
quantizes the data from transformer 1008 in a unit so 
of macro block. 

Step 113: Inverse quantizer 1010 inversely quan- 
tizes the data from quantizer 1009. 
Step 114: Inverse orthogonal transformer 1011 
inversely transforms the data from inverse quan- ss 
tizer 1010. 

Step 115: Frame decoder 1012 decodes a frame 
using the data inversely transformed with reference 



to reference-frame-buffer 1013 and motion -vector- 
buffer 1015. 

Step 116: Frame decoder 1012 stores data into 
buffer 1013 in order to estimate a motion. 
Step 117: Variable length encoder 1016 codes the 
data from quantizer 1009 in a variable length man- 
ner based on the quantization characteristics. 
Step 1 18: Input identifier 1007 identifies whether a 
frame to be coded still remains or not. If remains, 
identifier 1007 reads the frame data created in 
frame editor 112 (Step 103), and repeat the steps 
thereafter. If no frame remains, end the process. 

[0049] As such, the frame data supplied is coded by 
changing the quantization characteristics responsive to 
a data type based on the scenario information. 
[0050] The process of producing the helper-file (step 
101 and step 102) based on the scenario information is 
detailed hereinafter. The process of producing quantizer 
controlling file (steps 105, 106 and 107) from the helper- 
file by coding controller 1004 is also detailed hereinaf- 
ter. The controlling file is specified the quantiziation 
characteristics in each macro block shown in Fig. 16. 
[0051] First, the production process of the helper-file 
based on the scenario information is demonstrated, i.e. 
the steps 101 and 102 are detailed. 
[0052] As shown in Fig. 1 2. the data inside of the sce- 
nario comprises the following elements: 

(a) frame size information indicated in terms of 
lengths of X and Y axes; 

(b) data type information of respective sources; 

(c) time information indicated by a starting and end- 
ing times, describing frames where a source 
appears; and 

(d) a region indicated by coordinates of starting and 
ending points, describing a region within a frame, a 
source appears in the region. 

[0053] File creator 1 002 creates the helper-file based 
on the scenario information and the information which 
makes up the frame. This make-up information has 
been used for editing the frame discussed in the first 
embodiment. 

[0054] As shown in Fig. 15, the helper-file includes the 
following elements in each frame: 

(a) frame size information; 

(b) region information; 

(c) data type information of respective sources 
appearing each region; and 

(d) flag information indicating a frame where a 
source firstly appears or not, 

[0055] The helper-file is utilized for data-type-adap- 
tive-coding. 

[0056] As shown in Fig. 20, helper-file-creator 1002 
creates the helper-file based on the scenario shown in 



Step 101: Creator 1002 reads the scenario shown 
in Fig. 1 1 or detailed in Fig. 12. 10 
Step 102: Create a helper-file as shown in Fig. 15. 
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Fig. 12 and following the process shown below. 

Step 201: Add "1" to a frame counter which has 
indicated "0" at starting, thereby increasing a frame 
number by one. 

Step 202: Read a frame size and write it into the 
helper-file. 

Step 203: Read the information of the source firstly 
described in the scenario file. 
Step 204: Refer to the time information of the 
source, and return to step 103 when the present 
frame No. is not found between the starting and 
ending times. 

Step 205: When the present frame No. is found 

between the starting and ending times, determine 

whether the frame No. is "0" (zero) or not. 

Step 206: If the present frame No. is "0", write an 

appearing flag into the helper-file (Step 105). 

Step 207: Write the data type information into the 

helper-file. 

Step 208: Write the region information into the 
helper-file. 

Step 209: Determine whether a source to be proc- 
essed still remains or not. If remains, return to Step 
203, and if not, go to Step 210. 
Step 210: Determine whether a frame to be proc- 
essed still remains or not If remains, return to Step 
203. and if not, end the process. 

[0057] Next, the process of producing quantizer con- 
trolling file (steps 105, 106 and 107) from the helper-file 
by coding controller 1004 is demonstrated hereinafter. 
The controlling file is allotted with the quantiziation char- 
acteristics in each macro block shown in Fig. 16. 
[0058] Coding controller 1004 refers to data-type- 
adaptive-quantization-characteristics as shown in Fig. 
17 based on the helper file, and creates a macro block 
ID as well as a quantizer controlling file having quantiza- 
tion characteristics responsive to the macro block ID 
which is used for putting an ID number to a macro block. 
[0059] Fig. 21 illustrates the following process where 
coding controller 1004 creates a quantizer controlling 
file from the helper file. 

Step 301 : Add "1" to the frame counter which has 
indicated "0" at starting so that the counter number 
agrees with the frame number 
Step 302: From the helper-file, read the frame size 
information of the frame of which number agrees 
with the counter number. 

Step 303: Add "1" to a macro block counter— count- 
ing a macro block ID of which starting time is indi- 
cated "0" — and then write the result into quantizer 
controlling file. 

Step 304: Based on the frame size information 
read-in, determine the coordinates of macro blocks 
responsive to each macro block ID. 
Step 305: From the helper-file, read the region 



information within the frame of which No. ac ees 
with the frame counter number. 
Step 306: Refer to the region information, and 
determine whether the macro block is in the region 

s or not. 

Step 307: When the macro block is within the 
region, coding controller 1004 reads data-type 
information from the helper-file. 
Step 308: Refer to the data-type-adaptive-quantiza- 

70 tion-characteristics shown in Fig. 17. 

Step 309: Based on the characteristics, write the 
quantization characteristics responsive to the data- 
type information read-in at Step 307 into the quan- 
tizer controlling file. 

is Step 310: When the macro block is outside the 
region, determine whether other region information 
is available in the frame of which frame No. agrees 
with the frame counter No or not. If available, return 
to Step 305. 

20 Step 31 1 : When the region information responsive 
to the macro block does not exist, determine the 
data-type to be background, and refer to the quan- 
tization characteristics. 

Step 312: Based on the data-type-adaptive-quanti- 
25 zation characteristics, write the quantization char- 
acteristics in the case that the data-type is used as 
background into the quantizer controlling file. 
Step 313: Determine whether other macro blocks 
are available in the regon, and when available, 
30 return to Step 303. When not available, end the 
process. 

[0060] As such, the second embodiment has 
described an advantage as follows. 

35 [0061 ] A quantizer receives a quantizer controlling file 
based on an analysis of a scenario, and refers to a 
quantizer controlling file as well as a macro block ID that 
has undergone an orthogonal transforming, and thereby 
quantizing data with an optimal qunatization character- 

40 istics. As a result, a source can be coded adaptively to 
its data type in an efficient manner. 
[0062] In other words, a region where a still picture 
appears is quantized for a high quality picture. A region 
where text data appear is roughly quantized with a large 

45 quantization characteristics because color-difference- 
data is not important. Background is also roughly quan- 
tized because it is often painted with simple images 
having similar colors. A region where graphics appear is 
quantized with qunatization characteristics adaptive to a 

so rough picture because graphics are a kind of still pic- 
ture, drawn artificially and painted with a few colors. 
[0063] As such, since the data are quantized adap- 
tively to respective data types, a still picture — its picture 
quality is essential— turns out a quality picture even it 

55 has the same bit rate as other types of data. 
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Exemplary Embodiment 3 

[0064] An apparatus used in the third embodiment has 
the following features. 

[0065] When scenario-information indicates that the s 
first frame of a sequence of frames where a source 
appears is not an intra-frame, data-type-adaptive- 
encoder 1003 codes a region of the frame where the 
source appears as an intra-block 

[0066] A construction of the apparatus used in the 10 
third embodiment is the same as the construction that of 
the second embodiment shown in Fig. 2. Fig. 22 illus- 
trates a structure of encoder 1003, of which elements 
are the same as those in Fig. 13. Input identifier 1007 
used in this third embodiment selects and identifies the 75 
block of a region where the source appears— the block 
to be coded an intra-block — when scenario-information 
indicates that the first frame of a sequence of frames 
where a source appears is not an intra-frame. 
[0067] The data-type-adaptive coding apparatus hav- so 
ing the construction discussed above is demonstrated 
hereinafter with reference to Figs. 23 and 24. 
[0068] The process in scenario analyzer 1001 is the 
same as described in the second embodiment shown in 
Fig. 18. 25 
[0069] The process of this third embodiment is real- 
ized in this manner, i.e. data-adaptive-coding-controller 
1004 controls identifier 1007 based on the data-adap- 
tive-coding-helper-file described in Fig. 15, and the file 
is created by scenario analyzer 1001 . 30 
[0070] First, based on the helper-file created by ana- 
lyzer 1001, the data-type-adaptive-coding section 
codes the source adaptively to its data-type. The proc- 
ess flowchart in the coding section is described in Fig. 
23 with reference to Fig. 22. 35 

Step 403: Read frame-data created by frame editor 
1 1 2 out of frame buffer 1 13. 
Step 404: Store the data read in at Step 403 into 
input-frame-buffer 1006. 40 
Step 405: Controller 1004 reads the helper-file. 
Step 406: Controller 1004 creates an intra-f lag- 
information-file shown in Fig. 25 based on the 
helper-file shown in Fig. 15. 

Step 407: Controller 1004 transmits the intra -flag- 45 

information-file to identifier 1007. 

Step 408: Identifier 1007 identifies a kind of frames 

created. 

Step 409: When the frame created is B or P frame, 
identifier 1007 — by referring to the intra-f lag-infor- so 
mation-f ile shown in Fig. 25 and transmitted by con- 
troller 1004 — distinguishes macro-blocks 
estimating a motion from macro-blocks to be coded 
intra-blocks. 

Step 410: In the case of the macro blocks estimat- 55 
ing a motion, motion estimator 1014 estimates a 
movement with reference to frame data read by 
estimator 1014 into input-frame-buffer 1006 as well 



as frame data stored in reference buffer 1013. 
Step 411: Estimator 1014 stores motion-vector- 
information into motion -vector-buffer 1015. 
Step 412: (1) When the frame created is l-frame, 
the data transmitted by identifier 1007 undergoes 
orthogonal-transform. 

[0071] (2) When the frame created is B or P frame, 
and (a) in the case of a macro-block to be coded into an 
intra-block, orthogonal transformer 1008 transforms the 
data transmitted by identifier 1007, (b) in the case of a 
macro-block estimating a motion, transformer 1008 
transforms differential motion vector found by estimator 
1014. 

Step 413: Quantizer 2009 quantizes the data trans- 
mitted by transformer 1008. 
Step 414: Inverse quantizer 1010 inversely quan- 
tizes the data transmitted from data-type-adaptive 
quantizer 1009. 

Step 415: Inverse orthogonal transformer 1011 
inversely and orthogonal transforms the data trans- 
mitted from inverse quantizer 1010. 
Step 416: Frame decoder 1012 decodes the frame 
from the data inversely and orthogonal transformed 
with reference to buffers 1013 and 1015. 
Step 417: Frame decoder 1012 stores the data into 
buffer 1013 in order to estimate a motion. 
Step 418: Variable length encoder 1016 codes the 
data transmitted from qunatizer 2009 in a variable 
length manner. 

Step 419: Identifier 1007 checks a frame to be 
coded still remains or not. If remains, identifier 1007 
reads the frame data created in frame editor 112 
(Repeat Step 403 and onward). If not, end the proc- 
ess. 

[0072] Second, the detail process is described,' where 
data-type-adaptive-coding-controller 1004 creates the 
intra-flag-information-file out of the helper-file shown in 
Fig. 15, i.e. Step 405 and Step 406 are detailed herein- 
after. 

[0073] Out of the helper-file, controller 1004 creates 
the intra-f lag -information -file, which includes the follow- 
ing data: (1) a macro-block ID for numbering the macro- 
block, and (2) the data distinguishing the macro blocks 
to be motion-estimated as an intra-flag from the macro 
blocks to be coded as an intra-flag, this block is named 
"rand that block "0". 

[0074] Third, a process where controller 1 004 creates 
the intra-flag-information-file out of the data-type-adap- 
tive-coding-helper-file is demonstrated with reference to 
Fig. 24. 

Step 501 : Add "1 " to a frame counter which shows 
"0" at starting so that a frame number agrees with 
the counter number. 

Step 502: Add "1" to a macro-block-counter which 
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counts a macro-block ID and shows "0" at starting. 
Then write the result into the intra-f lag-information. 
Step 503: From the helper-file, read frame-size- 
data of which frame number agrees wittt the coun- 
ter number. 

Step 504: Determine the coordinates of the respec- 
tive macro-blocks responsive to each macro-block 
ID based on the frame-size data read in. 
Step 505: From the helper-file, read a piece of 
region information within the frame of which frame 
number agrees with the counter number. 
Step 506: Determine whether the macro-block is 
within the region or not by referring to the region 
information. 

Step 507: When the macro-block is within the 
region, read flag-information which indicates a 
frame where a source firstly appears from the 
helper-file. 

Step 508: Write the flag-information into the intra- 
flag-information-file. 

Step 509: When the macro-block is out of the 
region, determine whether another region informa- 
tion is in the frame of which frame number agrees 
with the counter number or not. If the region infor- 
mation exits, return to Step 505. 
Step 510: If another region information does not 
exist, determine that the macro-block is a back- 
ground, and write the information— the data did not 
appear firstly in this frame — into the intra-flag-infor- 
mation-file. 

Step 511 : Determine whether another macro-block 
exists or not. If exists, return to Step 503, and if not 
end the process. 

[0075] This third embodiment as discussed above 
proves the following advantage. Among a sequence of 
frames where a source appears, the first fame is not an 
intra-frame, the region where the source appears is 
coded as the intra-block. Although this frame thus has a 
higher probability that an image totally different from 
that of the previous frame appears, amount of calcula- 
tion is reduced than it is required in estimating a motion, 
and also picture quality is improved. 
[0076] Among a sequence of frames where a source 
appears, input identifier 1007 of the present invention 
codes the first frame into I -frame. 
[0077] On the other hand, among a sequence of 
frames where a source appears, input identifier 1007 of 
the present invention codes the first frame into P frame 
and codes the region, where the source is displayed, 
into an intra-block. 

[0078] Among a sequence of frames where a source 
appears, when the first frame is B frame, the region of B 
frames up to coming intra-frame or P frame can be 
coded as intra-blocks. In this case, when the frame 
other than B frame after the first frame is P frame, the 
region where the source appears is coded as an intra- 
block. 



Exemplary Embodiment 4 

[0079] An apparatus used in the fourth embodiment 
codes a source based on scenario information as fol- 
5 lows. 

[0080] (1 ) Frames where a still picture and/or text data 
are displayed, i.e. data free from time-varying is dis- 
played: 

Code the macro-blocks— making-up the region 
io where the source appears— as the macro-blocks at the 
same location in the previous frame. Escape informa- 
tion is thus indicated that these macro-blocks are identi- 
cal to those of the previous frame, and these macro- 
blocks are treated escape-macro-blocks. 
15 [0081 ] An entire construction of the apparatus used in 
this fourth embodiment is the same as that described in 
the second embodiment. 

[0082] The elements of data-type-adaptive-coding 
section shown in the block diagram in Fig. 26 are the 

20 same those in Fig. 13. 

[0083] Input identifier 1 007 used in this fourth embod- 
iment specifies the macro-blocks in the region where 
the data of the frames to be escape-macro-blocks indi- 
cating that these blocks have the same data as those 

25 macro-blocks at the same location of the previous 
frame. These frames can be specified during the period 
any frames— following the first frame among a 
sequence of frames where a source such as still picture 
or text data free from time-varying appears — is dis- 

30 played. 

[0084] An operation of the data-type-adaptive coding 
apparatus having the construction discussed above is 
demonstrated hereinafter with reference to Figs. 26, 27 
and 28. 

35 [0085] The process flowchart of scenario analyzer is 
the same as discussed in the second embodiment and 
shown in Fig. 18. 

[0086] Data-type-adaptive-coding-controller 1004 
shown in Fig. 22 controls input identifier 1007 based on 
40 a data-type-adaptive-coding-helper-file, thereby achiev- 
ing the process of this fourth embodiment. 
[0087] The process flowchart by a data-type-adaptive- 
coding section is demonstrated with reference to Fig. 26 
and Fig. 27. 

45 

Step 603: From frame-buffer 113, read frame-data 
created in frame editor 1 12. 
Step 604: Store the data read in at Step 603 into 
input-frame-buffer 1006. 
so Step 605: Controller 1004 reads the helper-file. 

Step 606: Controller 1004 creates an escape-infor- 
mation-file shown in Fig. 29 based on the helper-file 
shown in Fig. 15. 

Step 607: Controller 1004 transmits the escape- 
55 information-file to identifier 1007. 

Step 608: Identifier 1007 identifies a kind of frames 
created. 

Step 609: When the frame created is B or P frame, 
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identifier 1007 — by referring to the escape-infor- 
mation-file shown in Fig. 29 and transmitted by con- 
troller 1004 — distinguishes macro-blocks to be 
motion-estimated from macro-blocks to be coded 
as escape-macro-blocks. 5 
Step 610: In the case of the macro blocks to be 
motion-estimated, motion estimator 1014 estimate 
a motion with reference to frame data stored into 
input-frame-buffer 1006 as well as frame data 
stored in reference buffer 1013, i.e. estimator 1014 w 
calculates the difference between these two frame 
data. 

Step 611: Estimator 1014 stores motion-vector- 
information into motion-vector-buffer 1015. 
Step 612: (1) When the frame created is l-frame, is 
the data transmitted by identifier 1007 undergoes 
an orthogonal-transform. 



[0088] (2) when the frame created is B or P frame, and 
in the case of a macro-block to be motion-estimated, 20 
orthogonal transformer 1008 transforms orthogonal the 
data estimated by estimator 1014. 

Step 613: Quantizer 2009 quantizes the data trans- 
mitted by transformer 1 008. 25 
Step 614: Inverse quantizer 1010 inversely quan- 
tizes the data transmitted from data-type-adaptive 
quantizer 1009. 

Step 615: Inverse orthogonal transformer 1011 
inversely and orthogonal transforms the data trans- 30 
mitted from inverse quantizer 1010. 
Step 616: When B or P frame is to be created and 
is coded as an escape-macro-block, input identifier 
1007 transmits the information— the macro-block is 
an escape-macro-block— to both variable length 35 
encoder 1016 and frame decoder 1012. 
Step 617: Frame decoder 1012 refers to reference- 
frame-buffer 1013 and motion- vector-buffer 1015 
based on the escape-macro-block-information as 
well as the data inversely and orthogonal trans- 40 
formed, thereby restoring the frame. 
Step 618: Frame decoder 1012 stores the data into 
the reference buffer 1013 in order to estimate a 
motion. 

Step 619: Variable length encoder 1016 codes the 45 
two kinds of data in a variable length manner. One 
of the two kinds of data is the escape information 
transmitted by identifier 1007, and the other one is 
the data orthogonal transformed as a macro-block 
free from escaping by transformer 1008 and trans- so 
mitted by quantizer 2009. These two kinds of data 
are coded provided the escape information is 
embedded into a header of the macro-block free 
from escaping, this macro-block firstly coming after 
the macro-block escaping. 55 
Step 620: Identifier 1007 checks a frame to be 
coded still remains or not. If remains, identifier 1007 
reads the frame data created in frame editor 1 12 



(Repeat Step 603 and onward). If not, end the proc- 
ess. 

[0089] Next, the detail process is described, where 
data-type-adaptive-coding-controller 1004 creates the 
escape-information-file out of the helper-file shown in 
Fig. 15, i.e. Step 605 and Step 606 are detailed herein- 
after. 

[0090] Out of the helper-file, controller 1004 creates 
the escape-information-file, which includes the following 
data: (1) a macro-block ID for numbering the macro- 
block, and (2) the data distinguishing the macro blocks 
estimating a motion as an escape-information-insertion- 
flag from the macro blocks to be coded as an escape- 
macro-block, this block is named "1" and that block "0". 
[0091 ] Then, a process where controller 1 004 creates 
the escape-information-file out of the data-type-adap- 
tive-coding-helper-file is demonstrated with reference to 
Fig. 27. 

Step 701 : Add "1" to a frame counter which shows 
"0" at starting so that a frame number agrees with 
the counter number. 

Step 702: Add "1 " to a macro-block-counter which 
counts a macro-block ID and shows "0" at starting. 
Then write the result into the escape information. 
Step 703: From the helper-file, read frame-size- 
data of which frame number agrees with the coun- 
ter number. 

Step 704: Determine the coordinates of the respec- 
tive macro-blocks responsive to each macro-block 
ID based on the frame-size data read in. 
Step 705: From the helper-file, read a piece of 
region information within the frame of which frame 
number agrees with the counter number. 
Step 706: Determine whether the macro-block is 
within the region or not by referring to the region 
information. 

Step 707: When the macro-block is out of the 
region, determine whether another region informa- 
tion is in the frame of which frame number agrees 
with the counter number or not. If the region infor- 
mation exits, return to Step 705. 
Step 708: When the macro-block is within the 
region, read data-type-information from the helper- 
file. 

Step 709: Determine whether the macro-block is in 
a region where the source free from time-varying 
exists or not by referring to the data-type informa- 
tion. 

Step 710: When the region has the source free from 
time-varying, reads the appearing-f lag-information 
from the helper-file. 

Step 711: By referring to the appearing-f lag-infor- 
mation, determine whether the data of the region 
where the macro-block exists is the first appear- 
ance or not. 

Step 712: In the case that the source firstly appears 
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in this frame and also the case that the source of 
time-varying-video appears in a region of this 
frame, write a flag free from escaping into the 
escape-information-file. 

Step 713: If the frame did not encounter the first s 
appearance of the source, and the frame does not 
have another region information, write an escaping 
flag into the escape-information-file. 
Step 714: Determine whether anther macro-block 
exists or not t if exists, return to Step 703, and if not, w 
end the process. 

[0092] This fourth embodiment as discussed above 
proves that the coding adaptive to the data-type can be 
practiced. When a region-4he sources such as a still is 
picture or text data free from time-varying is dis- 
played — is processed, unnecessary processes by 
wasted estimation of motions can be thus eliminated. 
[0093] Neither of orthogonal transformation, quantiza- 
tion, inverse orthogonal transformation, or inverse 20 
quantization is not practiced, thereby improving the 
process speed. 

[0094] The data coded turn out smaller than the data 
by estimating motions, thereby improving the coding 
rate. Since this embodiment utilizes the information of a 25 
previous frame, a picture of better quality than a picture 
by estimating motions can be obtained. 
[0095] In B frame, in case of coding macro-blocks of a 
region displaying the data free from time-varying— -such 
as a still picture or text data — as escape-macro-blocks, 30 
when a forward frame needed for predicting has a totally 
different image, information is inserted into the macro- 
blocks so that a frame can be predicted only in an back- 
ward direction, before the macro-blocks are coded. 
[0096] In B frame, in case of coding macro-blocks of a 35 
region displaying the data free from time-varying— 6uch 
as a still picture or text data— as escape-macro-blocks, 
when a backward frame needed for predicting has a 
totally different image, information is inserted into the 
macro-blocks so that a frame can be predicted only in 40 
an forward direction, before the macro-blocks are 
coded. 

Exemplary Embodiment 5 

45 

[0097] An apparatus used in the fifth exemplary 
embodiment employs a method of changing quantiza- 
tion matrix instead of the method of changing the quan- 
tization characteristics, which has been used in the 
second exemplary embodiment. Materials other than so 
those used in this method are the same as used in the 
second embodiment. 

[0098] The fifth embodiment differs from the second 
one in the following points. The content of the quantizer- 
controlling-file shown in Fig. 16 is changed into a con- ss 
tent compirsing a macro-block ID for numbering a 
macro-block with an ID No. and a quantization matrix ID 
responsive to the macro-block ID as shown in Fig. 30. 



The data-type-adaptive-qunatization-characteristics 
shown in Fig. 17 is changed into data-type-adaptive- 
quantization-matrix-information shown in Fig. 31. This 
matrix information has a plurality of matrixes adaptive to 
a single data-type. 

[0099] An operation of the apparatus having the con- 
struction discussed above is demonstrated hereinafter 
with reference to Fig. 32 and Fig. 33. The process flow- 
chart by scenario analyzer 1001 is the same as 
described in Fig. 18 of the second embodiment. 
[0100] Fig. 32 is a process flowchart by data-type- 
adaptive-coding-section 1003. The different points 
between the flowcharts shown in Fig. 32 and Fig. 19 are 
described hereinafter. 

[0101] Step 803 through Step 81 1 shown in Fig. 32 
are the same as Step 103 through Step 111 shown in 
Fig. 19, thus the descriptions of these steps are omitted 
here. 

Step 812: Quantizer 1009 has received the quan- 
tizer-controlling-file created by coding controller 
1004 shown in Fig. 16. Quantizer 1009 determines 
a quantization-matrix- ID in a macro-block unit 
responsive to a macro-block ID detailed later, and 
then quantizes the data— transmitted from orthogo- 
nal transformer 1008 — by using a quantization 
matrix responsive to the quahtization-matrix-ID 
which quantizer 1009 has already determined. 
Step 813 through 816 shown in Fig. 32 are the 
same as Step 113 through Step 116 shown in Fig. 
19. The descriptions of these steps are thus omit- 
ted here. 

Step 817: Variable-length-encoder 1016 the data 
transmitted from quantizer 1009 with reference to 
the quantization-matrix-ID. 

Step 818: Input identifier 1007 determines whether 
a frame to be coded still remains or not. If the frame 
remains, identifier 1007reads the frame data cre- 
ated in frame editor 112 (repeat Step 803 and 
onward), if not, end the process. 

[01 02] As such, a frame data supplied is coded adap- 
tively to its data type. When H is coded, the quantization- 
matrix is changed adaptively to the data type based on 
the information created by the scenario analyzer. 
[0103] Coding controller 1004 creates a quantizer- 
controlling-file out of a data-type-adaptive-coding- 
helper-file. This process is described hereinafter with 
reference to Fig. 33 in the different points from that of 
the second embodiment shown in Fig. 21 . 
[01 04] Steps 901 through 907 shown in Fig. 33 are the 
same as Steps 301 and 302 in Fig. 21 . The descriptions 
of these steps are thus omitted here. 

Step 908: Refer to the data-type-adaptive-quantiza- 
tion-matrix-information shown in Fig. 31. 
Step 909: Based on the information of data-type- 
adaptive-quantization-matrix shown in Fig. 31, write 



12 

BNSDOCID: <EP 0989563A2_I_> 



23 



EP 0 989 563 A2 



24 



the quantization matrix responsive to the data-type 
information read-in at Step 907 into the quantizer 
controlling file. 

Step 910: When the macro block is outside the 
region, determine whether other region information 
is available in the frame of which frame No. agrees 
with the frame counter No or not. If available, return 
to Step 905. 

Step 91 1 : When the region information responsive 
to the macro block does not exist, determine the 
data-type to be background, and refer to the infor- 
mation of the quantization matrix. 
Step 912: Based on the data-type-adaptive-quanti- 
zation characteristics, write the quantization 
matrix— in the case that the data-type is used as 
backgrorund— into the quantizer controlling file. 
Step 913: Determine whether other macro blocks 
are available in the region, and when available, 
return to Step 903. When not available, end the 
process. 

[0105] As such, the fifth embodiment has proved the 
following advantage. 

[01 06] A quantizer receives a quantizer controlling file 
based on an analysis of a scenario, and refers to a 
quantizer controlling file as well as a macro block ID that 
has undergone an orthogonal transforming, and thereby 
quantizing data with an optimal qunatization matrix. As 
a result, a source can be coded adaptiveiy to its data 
type in an efficient manner. 

[0107] Regarding a region where text data appears, a 
quantization matrix must be so prepared to value a 
high-frequency-component, i.e. coefficients of the 
region crowded with high-frequency-components must 
be lowered, and on the contrary, coefficients of the 
region crowded with low-frequency-components must 
be raised. Regarding a background, an artificial simple 
image, e.g. using only three colors such as graphics, is 
often used. Thus the quantization on the background 
employs a matrix where low-frequency-components are 
not valued. 

[0108] As such, quantization matrixes adaptive to 
respective data types can be employed at quantization, 
so that text data is displayed with clear image of high- 
frequency-components. 

[0109] A quantization matrix can be changed accord- 
ing to the ratio of region area of each source, i.e. a ratio 
among regions' areas where respective sources 
appear. 

Exemplary Embodiment 6 

[0110] An apparatus used in the sixth exemplary 
embodiment stores either one of the following two data 
into block-buffer 6017 shown in Fig. 34. (1) the first 
frame where the data of non-time-varying video such as 
a still picture or text data appears, or (2) the data 
obtained by quantizing the region in the first intra-frame, 



in the region the data of non-time-varying video 
appears. Among a sequence of frames where that data 
appears, the intra-frame does not undergo an orthogo- 
nal transform or quantization, but is coded by inserting 

5 the quantized data. 

[0111] An entire construction of the apparatus in 
accordance with the sixth embodiment is the same as 
that used in the second embodiment and shown in Fig. 
10. Fig. 34 is a block diagram of an inside structure of 

w data-type-adaptive coding-section 1003 shown in Fig. 
10. 

[0112] Block buffer 6017 stores the following data: (1) 
a first frame where the data of non-time-varying-video 
such as a still picture or text data appears, or the data 

is quantized of the region in an intra-frame, the region 
where the data of non-time-varying video appears, and 
(2) the data undergoes an inverse orthogonal transform 
and inverse quantization. Other elements are the same 
as shown in Fig. 13. 

20 [01 13] Input identifier 1007 used in this sixth embodi- 
ment utilizes block buffer 6017 based on the information 
supplied from coding -controller 1004, thereby distin- 
guishing the data undergone quantization in the region 
where the data of non-time-varying-video such as a still 

25 picture or text data appears. 

[0114] An operation of identifier 1007 is demon- 
strated. Identifier 1007 performs the following four 
items. (1) Quantizer 2009 quantizes the region where 
the data of non-time-varying-video such as a still picture 

30 or text data appears. Identifier 1007 receives the result- 
ant (quantized) data from quantizer 2009, and stores it 
into block buffer 6017. (2) Identifier 1007 transmits the 
data stored in buffer 6017 to variable-length-encoder 
1016 when intra-frames are displayed, i.e. during the 

35 period where the data of non-time-varying-video 
appears. (3) Identifier 1007 receives the data under- 
gone an orthogonal transform and inverse quantization 
from frame<lecoder 1012, and stores the same data 
into block buffer 6017. (4) Identifier 1007 transmits the 

40 same data stored in buffer 601 7 to frame-decoder 1 012 
when intra-frames are decoded among the frames 
where the data of non-time-varying-video such as a still 
picture or text data appears. 

[01 15] As such the apparatus in accordance with the 
45 sixth embodiment proves the following advantage. In an 
intra-frame among the frames where the data of non- 
time-varying-video such as a still picture or text data 
appears, neither an orthogonal transform, quantization, 
inverse quantization nor inverse orthogonal transform is 
so practiced, so that a process has been completed in a 
shorter time. No difference due to quantization methods 
occurs. As a result, picture quality is improved. 
[01 16] A combination of the function of encoder 1 005 
of this sixth embodiment with that of the third embodi- 
55 ment can be applied in the following case. i.e. in frames 
other than intra frames, the data of non-time-varying- 
video such as a still picture or text data appears. 
[0117] rf the function of encoder 1005 of this sixth 
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embodiment is combined with the function of the 
encoder of the fourth embodiment, the data stored in 
the block-buffer can be displayed free from any changes 
in video from the first intra-frame where the data 
appears. This video can be kept free from any changes § 
in video even when B or P frame is displayed. Thus the 
data is not subject to GOP units, and the picture quality 
is improved. 

[0118] If the function of encoder 1005 of this sixth 
embodiment is combined with the function of the 10 
encoder of the fourth embodiment and that of the third 
embodiment, the data of non-time-varying-video is dis- 
playaWe without any changes in video during its display 
even when B or P frame is displayed. As a result, picture 
quality is improved. /5 

Exemplary Embodiment 7 

[0119] An apparatus in accordance with the seventh 
exemplary embodiment has the following features. 20 
Based on scenario information, when P or B frame is 
displayed, the apparatus inputs the information of "no 
change in the data" into P or B frame in every DCT 
block, except when the data of non-time-varying-video 
such as a still picture or text data appears. 25 
[0120] The apparatus of producing coded video sig- 
nals in accordance with the seventh embodiment has 
the same structure as used in the fourth embodiment. 
An operation of this apparatus has the following per- 
formance in addition to the operation described in the 30 
fourth embodiment. Among macro-blocks, if there are 
two or more than two regions having different data- 
types respectively, and also there is a region-nn a DCT 
block unit— where the data of non-time-varying-video 
such as a still picture or text data appears, the informa- 35 
tion indicating a block-coding-pattern such as coded- 
block-pattern (CBP) in MPEG, whereby the information 
of "no change in the data" is supplied in every DCT 
block. This performance is added to the operation in the 
fourth embodiment. 40 
[0121] In the case of a macro-block shown in Fig. 35, 
for instance, regions shown in Fig. 36 exist in the macro- 
block. In this case, three blocks out of four DCT blocks 
are background regions which are still pictures forming 
a background. The information of "no change in every 45 
frame of these blocks" can be supplied to the macro 
block. For example, the information of coding patterns 
such as CBP is added to the macro-block. The coding 
pattern indicates "no change" in the three DCT blocks 
making up the background shown in Fig. 36. so 
[01 22] As such, the seventh embodiment has demon- 
strated the advantages as follows: While the data of 
non-time-varying-video is displayed in every DCT block, 
the data can be free from changes in video. In the back- 
ground of a motion picture as shown in Fig. 35. where a ss 
border between the data of non-time-varying and the 
data of time-varying is formed, the picture quality of 
non-time-varying-data is thus improved. 



[0123] K the function of the data-type-adaptive- 
encoder in accordance with the seventh embodiment is 
combined with that of the fourth embodiment, the 
improved picture quality of non-time-varying-data as 
discussed above can be expected in addition to the 
advantage obtained by the fourth embodiment. 

Exemplary Embodiment 8 

[0124] An apparatus used in the eighth exemplary 
embodiment estimates a motion adaptive to its data- 
type in an efficient manner based on a scenario, where 
coding controller 1004 shown in Fig. 37 determines a 
search area of a motion estimator. 
[0125] An entire construction of the apparatus in 
accordance with the eighth embodiment is the same as 
that used in the second embodiment and shown in Fig. 
10. Fig. 37 is a block diagram of an inside structure of 
data-type-adaptive coding-section 1003 shown in Fig. 
10. 

[0126] Data-type-adaptive-motion-estimator 8014 is 
controlled by coding controller 1004 and estimates a 
motion adaptive to its data-type. Other elements are the 
same as shown in Fig. 13. 

[0127] An operation of estimator 8014 is demon- 
strated hereinafter. If the search area for a macro-block 
is defined as shown in Fig. 38. coding controller 1004 
utilizes the information narrowing down the search area 
shown in Fig. 39, whereby estimator 8014 estimates a 
motion within the narrowed search area. 
[0128] As such, the eighth embodiment has proved 
the advantage as follows: When a search area of motion 
estimation is defined as shown in Fig. 38. the search 
area is narrowed down as small as shown in Fig. 39. 
The process thus can be reduced and takes a shorter 
time. Estimation errors are also reduced, and as a 
result, picture quality is improved. 
[01 29] The search area can be dislocated as shown in 
Fig. 40, so that more precise motion-estimation is prac- 
ticed. Then picture quality is improved. 

Exemplary embodiment 9 

[0130] If MPEG 2 method is taken as an example, a 
quantization matrix is changeable only with a frame unit. 
However, an apparatus in accordance with this ninth 
exemplary embodiment utilizes quantization matrixes 
adaptive to respective data-types within the same 
frame, and thereby coding the data. For instance, for 
text data section, quantization matrixes adaptive to text 
data are used, and for a motion picture section, quanti- 
zation matrixes optimum to motion picture are used. 
[01 31 ] A specific method is demonstrated hereinafter. 

(1) In the case that text data appears firstly on P or I 
frame: 

[0132] 
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(1-a) Code the macro-block as an intra-block, the 
macro-block includes a region where text-data 
appears, and the region is in the first P frame, e.g. 
4007 frame, following the infra-frame as shown in 
Fig. 41. For this purpose, the data is quantized with 
a quantization matrix for intra-picture adaptive to 
text data. In regions other than the regions dis- 
cussed above, the data are quantized by using 
quantization matrixes (non-intra-quantization- 
matrix) for differential motion vector found through 
the motion estimation. 

(1 -b) Between the P frame and the next intra-frame, 
escape information of "keeping as previous" is 
inserted in the region on P or B frames. On these 
frames, the same videos as shown in P frame dis- 
cussed in (1 -a) are displayed. 
(1-c) In the intra-frame, the data inclusive text data 
are quantized with quantization matrix for a regular 
motion picture (default quantization matrix) adap- 
tive to the entire frame. 

(1-d) If B frame exists between the intra-frame and 
the P frame following the intra-frame, an estimation 
is arranged for B frame so that the coming P frame 
after the intra-frame only estimates a macro-block 
of the region where text data appears. 

[01 33] This series of performances are repeated dur- 
ing the display of text data, whereby the regions where 
text data appears except the intra-frames are displayed 
with the data undergone the quantization adaptive to 
text data. 

[0134] When the first frame on which text data 
appears is an intra-frame, the same performance dis- 
cussed in (1-c) is practiced. 

(2)-1 . In the case that text data appears firstly on B 
frame, and the next frame except B frame is P frame: 

[0135] 

(2)-l -a. Code the macro-block as an intra-block, the 
macro-block includes a region where text-data 
appears, and the region is in P frame, e.g. 4007 
frame, following the second intra-frame (4005) as 
shown in Fig. 41 . For this purpose, the data is quan- 
tized with an infra-purpose quantization matrix 
adaptive to text data. In regions other than the 
above mentioned regions, the data are quantized 
by using quantization matrixes (non-intra-quantiza- 
tion-matrix) for differential motion vector found 
through the motion estimation. 
(2)-1-b. Regarding B frames from the first B frame 
where text data appears firstly to the next P frame, 
an estimation is arranged for these B frames so that 
the coming P frame after the infra-frame only esti- 
mates a macro-block of the region where text data 
appears. 



[0136] The regions where text data appear are thus 
displayed with the data undergone the quantization 
adaptive to text data. 

5 (2) -2. In the case that text data appears firstly on B 
frame, and the next frame except B frame is I frame: 

[0137] Regarding B frames from this first B frame to 
next I frame, code the macro-block as an intra-block, the 

w macro-block includes a region where text-data appears. 
In this case, a quantization matrix for intra-picture adap- 
tive to text data is used. Sources other than text data are 
quantized by using quantization matrixes (non-intra- 
quantization-matrix) for differential motion vector found 

is through the motion estimation. 

[0138] The data quantized as the infra-block where 
text data appears are stored in the block-buffer, and this 
data can be copied for the next infra-frame, thereby 
reducing a volume of calculation. 

so [0139] An entire construction of the apparatus in 
accordance with the ninth embodiment is the same as 
that used in the second embodiment and shown in Fig. 
10. Fig. 42 is a block diagram of an inside structure of 
data-type-adaptive coding-section 1003. 

25 [0140] Block buffer 6017 stores the following data: (1) 
the data quantized as an infra-block in a first frame 
where text data appears, and (2) the data undergoes an 
inverse orthogonal transform and inverse quantization. 
Other elements are the same as shown in Fig. 13. 

30 [01 41 ] Input identifier 1 007 used in this ninth embodi- 
ment utilizes block buffer 6017 based on the information 
supplied from coding-controller 1004, thereby utilizing 
efficiently the data quantized as the infra-block at 
appearing of text data. 

35 [0142] In the region, macro-blocks where sources 
other than intra-frames and intra-blocks appear are 
specified as escape-macro-blocks where data are free 
from changes. 

[01 43] An operation of input identifier 1 007 used in the 
40 ninth embodiment is demonstarated. Identifier 1007 
practices the following jobs. 

[0144] (1) Quantizer 2009 quantizes the region where 
text data appears. Identifier 1007 receives the resultant 
(quantized) data from quantizer 2009, and stores it into 

45 block buffer 601 7. (2) Identifier 1007 transmits the data 
stored in buffer 6017 to variable-length-encoder 1016 
when infra-frames are displayed, i.e. during the period 
where text data appears. (3) Identifier 1007 receives the 
data undergone an orthogonal transform and inverse 

so quantization from frame-decoder 1012, and stores the 
same data into block buffer 6017. (4) Identifier 1007 
transmits the same data stored in buffer 6017 to frame- 
decoder 1012 when P frame next to the infra-frames is 
decoded while text data appears. (5) While sources 

55 other than text data are displayed, the macro-blocks in 
the region other than the infra-frames and intra-blocks 
are coded as escape-macro-blocks. 
[0145] As such the ninth embodiment has proved the 
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following advantage: If MPEG 2 method is taken as an 
example, a quantization matrix is changeable only with 
a frame unit. However, this ninth exemplary embodi- 
ment utilizes quantization matrixes adaptive to respec- 
tive data-types within the same frame, and thereby 5 
coding the data. In actual, for text data section, quanti- 
zation matrixes adaptive to text data are used. As a 
result, picture quality of the region where text data is 
displayed is improved. 

[0146] This advantage is not limited to text data, but 10 
applicable to non-time-varying data, for which the quan- 
tization matrix adaptive to this data-type is utilized, and 
the data is thus efficiently coded. 
[0147] To be more specific, regarding a region where 
text data appears, a quantization must be so prepared 75 
to value a high-frequency-component, i.e. coefficients 
of the region crowded with high-frequency-components 
must be lowered, and on the contrary coefficients of the 
region crowded with low-frequency-components must 
be raised. 20 
[0148] Regarding a background, an artificial simple 
image, e.g. using only three colors such as graphics, is 
often used. Thus the quantization on the background 
employs an unbalanced matrix where low-frequency- 
components are not valued. As such, quant zation 25 
matrixes adaptive to respective data types can be 
employed at quantization, so that e.g. text data is dis- 
played with clear image of high-frequency-components. 
[0149] In the exemplary embodiments discussed pre- 
viously, a process within a region of a single source has 30 
been discussed. However, the region of a single source 
can be divided into sections, and respective scenarios 
are equipped with relevant information. Each section 
can be coded based on respective information and 
adaptively to data-types of each section. 35 
[0150] The data obtained by the apparatus and 
method is storable in a recording media. 

Claims 

40 

1 . A multimedia information coding apparatus for edit- 
ing a multimedia title comprising a source including 
a motion picture, a still picture, text data as well as 
audio and a scenario of the source into a video sig- 
nal, and for coding the video signal, said apparatus 45 
comprising: 

(a) a source memory for storing a source; 

(b) a scenario memory for storing a scenario; 

(c) a frame editor for creating a frame of one so 
video stream by using the scenario and the 
source stored in said source memory 

(d) a motion picture encoder for coding the 
frame edited by said frame editor. 

55 

2. The multimedia information coding apparatus as 
defined in Claim 1 further comprising: 



(e) an audio component memory for storing an 
audio component out of the source; 

(f) a frame buffer for storing the frame edited by 
said frame editor; 

(g) an audio encoder for coding audio stored in 
said audio component memory; and 

(h) a multiplexer for multiplexing a motion pic- 
ture and audio both having been coded. 

3. The multimedia information coding apparatus as 
defined in Claim 2 wherein said source memory 
comprising: 

a motion picture memory for storing a motion 
picture source; 

a still picture memory for storing a still picture 
source; and 

a text data memory for storing a text data 
source. 

4. The multimedia information coding apparatus as 
defined in Claim 1 wherein the source is preparato- 
rily converted into a component having a same for- 
mat as the video signal to be edited, before the 
multimedia title comprising the source and scenario 
is edited to the video signal. 

5. The multimedia information coding apparatus as 
defined in Claim 3 further comprising: 

a motion picture decoder for decoding a motion 
picture source including at least one of a 
motion picture and audio; 
a still picture decoder for decoding a still picture 
into a frame component having a same signal 
format as the video signal to be edited; 
a text data decoder for decoding text data into 
a frame component having a same signal for- 
mat as the video signal to be edited; and 
a frame component memory for storing the 
frame component 

wherein said frame editor creates one video 
stream by using the scenario, the frame com- 
ponent stored in said frame component mem- 
ory and a decoded motion picture source. 

6. The multimedia information coding apparatus as 
defined in Claim 5 wherein the video signal is edited 
based on frame-structure-information describing 
source information for editing the video signal 
responsive to each frame. 

7. The multimedia information coding apparatus as 
defined in Claim 5 further comprising a scenario 
analyzer for creating frame-structure-informa- 
tion — responsive to each frame of the video signal 
to be edited — describing source information 
needed for editing out of the scenario, and 
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wherein said frame editor creates one video stream 
out of the frame components, the decoded motion 
picture source based on the frame-structure-infor- 
mation. 

5 

8. A multimedia information coding apparatus for cod- 
ing a multimedia title adaptively to respective data- 
type of each source based on scenario information, 
said apparatus comprising: 

w 

a scenario analyzer for analyzing a scenario; 
and 

a data-type-adaptive-coding section for coding 
data adaptively to own data type based on the 
scenario information analyzed. is 

9. The multimedia information coding apparatus as 
defined in Claim 6 wherein said scenario analyzer 
creates data-type-adaptive-coding-helper-informa- 
tion including for each frame: 20 

region information describing which region the 
source appears; 

time information describing when the source 
appears; and 25 
data-type-information describing the data-type 
of the source 

wherein said data-type-adaptive-coding-sec- 
tion includes a data-type-adaptive-encoder and 
a data-type-adaptive-coding-controller which 30 
controls the data-type-adaptive-encoder, using 
the created helper-information. 

10. The multimedia information coding apparatus as 
defined in Claim 9 wherein said data-type-adaptive- 35 
encoder comprising: 

(a) an input identifier for identifying a frame 
data adaptive to a coding format of the frame 
data supplied; 40 

(b) an orthogonal transformer for providing the 
supplied frame data with DCT process; 

(c) a quantizer for quantizing the frame data 
undergone orthogonal transform; 

(d) a motion estimation means for estimating a 45 
motion and outputting a motion vector data 
based on the frame data supplied and the 
quantized data quantized by said quantizer; 

(e) a variable length encoder for coding both 
the frame data quantized and the motion- vec- so 
tor-data, 

wherein said motion estimating means refers to 
the coding format of the frame data identified 
by said input identifier. 

55 

11. The multimedia information coding apparatus as 
defined in Claim 10 wherein said quantizer quan- 
tizes the frame data based on a process mode 



determined by the data-type-adaptive-coding-con- 
troller. 

12. The multimedia information coding apparatus as 
defined in Claim 1 1 wherein the data-type-adap- 
tive-coding-controller creates quantization contor- 
lling information having region-oriented 
quantization-characteristic-information adaptive to 
the data-type based on the helper-information, and 
determines quantization characteristics utilized by 
said quantizer. 

13. The multimedia information coding apparatus as 
defined in Claim 12 wherein quantization character- 
istics for a region where a still picture appears are 
determined to be the values for a high quality pic- 
ture. 

14. The multimedia information coding apparatus as 
defined in Claim 1 2 wherein quantization character- 
istics for a region where a still picture appears are 
determined to be the values for a rough picture. 

15. The multimedia information coding apparatus as 
defined in Claim 9 wherein when the data-type 
information indicates a frame where the source 
appears is any frame other than an intra-frame, a 
region where the source appears is coded as the 
intra-block. 

16. The multimedia information coding apparatus as 
defined in Claim 10 wherein said input identifier 
identifies a coding format of the supplied frame data 
to be an intra-frame and a frame other than an intra- 
frame , and 

wherein when a frame where a source of the 
supplied data has appeared is any frame other than 
the intra-frame, the identifier identifies a block to be 
coded as an intra -block for coding a region where 
the source has appeared as an intra-block. 

17. The multimedia information coding apparatus as 
defined in Claim 10 wherein a block of a region 
where a source of non-time-varying- video appears 
is coded by using data of coding a block corre- 
sponding to said block of a frame which has 
appeared before the frame including said block. 

18. The multimedia information coding apparatus as 
defined in Claim 1 0 wherein when data of non-time- 
vary ing-video appears in the supplied data, a 
macro-block within a display area of a frame other 
than a frame where the data has firstly appeared is 
coded as an escape-macro-block. 

19. The multimedia information coding apparatus as 
defined in Claim 10 wherein based on the helper- 
file, said data-type-adaptive-coding-controller ere- 
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ates quantization-corrtrolling-infonmation having 
quantization-matrix-information adaptive to a data- 
type and for each region, and 

wherein said quantizer is controlled based 
on the quantization-controlfing-information. 5 

20. The multimedia information coding apparatus as 
defined in Claim 10 wherein said scenario analyzer 
allots each data-type to respective regions accord- 
ing to an area ratio of respective regions where 10 
each source appears, and 

wherein said quantizer quantizes data by 
changing a quantization matrix based on the allot- 
ment o1 the data-types. 

15 

21. The multimedia information coding apparatus as 
defined in Claim 10 wherein for each frame and 
based on the helper-file, the coding controller allots 
each data-type to respective regions according to 

an area ratio of respective regions where each 20 
source appears, and the coding controller creates 
quantization-controlling-information having quanti- 
zation-matrix-information of each frame, and 

wherein said quantizer quantizes data based 
on the quarrtization-controlling-information. 25 

22. The multimedia information coding apparatus as 
defined in Claim 9 further comprising a buffer for 
storing coded data of a region where a source 
appears, the region is in an intra-frame where a 30 
non-time-varying-source appears firstly, wherein 
when an intra-frame exists in a sequence of the 
frames where the source is kept displaying, the 
data stored in said buffer is copied on the intra- 
frame. 35 

23. The multimedia information coding apparatus as 
defined in Claim 10 further comprising a block 
buffer, and 

said motion estimating means further comprising 40 

(i) inverse quantizer for inversely quantizing the 
frame data undergone quantization; 

(ii) inverse orthogonal transformer for providing 

the frame data undergone inverse quantization 45 
with inverse DCT process, then creating a non- 
coded video signal; and 

(iii) frame decoder for decoding a frame from 
the frame data undergone inverse orthogonal 
transform, 50 
wherein said input identifier produces the fol- 
lowing results: 

(a) identifying a coding format of the sup- 
plied-frame-data to be an intra-frame, a bi- 55 
directional predictive frame and a predic- 
tive frame; 

(b) receiving from said quantizer quantized 
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data obtained by coding data in the sup- 
plied data as a first irrtra-block. and storing 
the received data into said block buffer; 

(c) transmitting the data stored in the block 
buffer to a variable-length-encoder at an 
intra-frame while data of non-time-varying- 
video including a still picture and text data 
appears; 

(d) receiving from a frame-decoder data 
undergone inverse orthogonal transform at 
said inverse orthogonal transformer and 
inverse quantization at said inverse quan- 
tizer, and stores the data into the block 
buffer; and 

(e) transmitting the data stored in the block 
buffer and received from the frame- 
decoder to the frame decoder when an 
intra-frame is decoded during data of non- 
time-varying-video being displayed. 

24. The multimedia information coding apparatus as 
defined in Claim 10 further comprising a block 
buffer, and said motion estimating means further 
comprising a motion estimating means further com- 
prising 

(i) an inverse quantizer for inversely quantizing 
the frame data undergone quantization; 

(ii) an inverse orthogonal transformer for pro- 
viding the frame data undergone inverse quan- 
tization with inverse DCT process, then 
creating a non-coded video signal; and 

(iii) a frame decoder for decoding a frame from 
the frame data undergone inverse orthogonal 
transform, 

wherein sad input identifier produces the fol- 
lowing results: 

(a) identifying a coding format of the sup- 
plied-frame-data to be an intra-frame, a bi- 
directional predictive frame and a predic- 
tive frame; 

(b) reading the supplied data and based on 
a process mode determined by the coding- 
controller, when a source has firstly 
appeared in a frame other than an intra- 
frame, identifying a block to be coded as 
the tntra-block in a region the source 
appears in the frame; 

(c) among the supplied-data, receiving 
from said quantizer quantized -data of a 
region where non-time-varying-data such 
as a still picture and text data appears, and 
storing the received data into the block 
buffer; 

(d) transmitting the data stored in the block 
buffer to a variable-length-encoder at an 
intra-frame while data of non-time-varying- 
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video such as a still picture and text data 
appears; 

(e) receiving from a frame-decoder data 
undergone inverse orthogonal transform at 
said inverse orthogonal transformer and s 
inverse quantization at said inverse quan- 
tizer, and stores the data into the block 
buffer; and 

(f) transmitting the data stored in the block 
buffer and received from the frame- 10 
decoder to the frame decoder when an 
intra-frame is decoded during data of non- 
time-varying-video being displayed. 

25. The multimedia information coding apparatus as is 
defined in Claim 18 wherein when data of different 
data-types appear in the macro-block, and in a 
case of "no change from previous data" information 
can be inserted in a unit of DCT; quantization and 
DCT are eliminated by inserting information 20 
described with a coding pattern. 

26. The multimedia information coding apparatus as 
defined in Claim 10 wherein said motion estimation 
means is controlled by the coding-controller for esti- 25 
mating a motion adaptively to a data-type; and 
wherein a search area of motion estimation is 
defined based on information obtained by analyzing 

a scenario with said scenario analyzer. 

30 

27. The multimedia information coding apparatus as 
defined in Claim 10 wherein said input identifier 
identifies a coding format of the supplied-frame- 
data to be an intra-frame and a frame other than 
intra-frame; 35 

wherein said data-type-adaptive-encoder 
produces results as follows: 

(a) in a case that a frame data supplied is video 
data of one of text data and graphics: 40 

(a-1) when a frame where the video data 
firstly appears is an intra-frame, a macro- 
block in a region where the frame data is 
displayed is coded as an intra-block; 45 
(a-2) when the frame where the video data 
firstly appears is any frame other than an 
intra-frame, a macro-block in a region 
where the frame data is displayed is coded 
as an intra-block, and a quantization matrix so 
adaptive to data-type of the video data is 
used for the coding, 

(b) regions other than the region where the 
video data appears are quantized by using a ss 
quantization matrix for differential motion vec- 
tor obtained through the motion estimation. 



28. The multimedia information coding apparatus as 
defined in Claim 10 further comprising a block 
buffer, and said motion estimating means further 
comprising comprising a motion estimating means 
further comprising 

(i) an inverse quantizer for inversely quantizing 
the frame data undergone quantization; 

(ii) an inverse orthogonal transformer for pro- 
viding the frame data undergone inverse quan- 
tization with inverse DCT process, then 
creating a non-coded video signal; and 

(iii) a frame decoder for decoding a frame from 
the frame data undergone inverse orthogonal 
transform, 

wherein said input identifier produces results 
as follows: 

(a) identifying a coding format of the sup- 
plied -frame-data to be an intra-frame and a 
frame other than intra-frame; 

(b-1) when a frame where video data 
one of text data and graphics out of the 
supplied data appears firstly is an 
intra-frame, receiving from said quan- 
tizer quantized data of a region where 
a source of the next frame appears, 
and storing the quantized data into the 
block buffer; 

(b-2) when a frame where video data 
one of text data and graphics out of the 
supplied data appears firstly is any 
frame other than an intra-frame, 
receiving from said quantizer quan- 
tized data of a region where a source 
of the frame appears, and storing the 
quantized data into the block buffer; 

(c) transmitting the quantized data stored 
in the block buffer to said variable length 
encoder in coding a frame next to the intra- 
frame where the video data appears; 

(d) receiving from a frame-decoder data 
undergone inverse orthogonal transform at 
said inverse orthogonal transformer and 
inverse quantization at said inverse quan- 
tizer, and stores the data into the block 
buffer; 

(e) transmitting the data stored in the block 
buffer and received from the frame- 
decoder to the frame decoder when a 
frame next to the intra-frame is decoded 
during the video data being displayed; and 

(f) specifying a macro-block within a 
region — where the video data 
appears — as an escape-macro-block 
which displays a same video as a previous 
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frame does; said macro-block is in a frame 
other than a frame where the video data 
firstly appears and any block other than 
intra-frame and intra-block, and 
wherein said quantizer quantizes data 5 
based on a process mode determined by 
the coding controller. 

29. A method of coding multimedia information, the 
method coding a video signal edited out of a mufti- 10 
media title comprising a source including a motion 
picture, a still picture, text data as well as audio, and 

a scenario describing a source list, a presentation 
method of the source as well as characteristics, 
based on the scenario, said method comprising the is 
steps of: 

(a) reading the scenario; 

(b) based on information described in the sce- 
nario, creating information of the source mak- 20 
tng up a frame of an output video signal; 

(c) based on information described in the sce- 
nario, creating a frame component by convert- 
ing a source into one signal format; 

(d) editing one frame out of the frame compo- 25 
nent by referring to the information created in 
step (b); and 

(e) creating one video signal out of a frame 
comprising the frame edited in step (d). 

30 

30. The method of coding multimedia information as 
defined in Claim 29 wherein in step (c), the sources 
have been converted into frame components com- 
prising signals in a same format as the video signal 

to be edited. 35 

31. The method of coding multimedia information as 
defined in Claim 29 wherein the information created 
in step (b) includes frame-structure-information 
describing information about a source to be edited 40 
in each frame; and 

wherein in step (d), editing a frame based on the 
frame-structure-information. 

32. A method of coding multimedia information, the 45 
method coding each source adaptiveiy to respec- 
tive data-types based on scenario information, said 
method comprising the steps of: 

(a) reading the scenario; so 

(b) obtaining, from the scenario, information 
including region and time a data-type of the 
source is displayed; 

(c) based on the information obtained in step 
(b), creating information including the data-type ss 
of the source, region and time the data-type of 
the source is displayed for each frame; 

(d) based on the information created in step (c), 



38 

coding supplied data of a multimedia title for 
each frame by using at (east one of following 
three steps: 

(d-1) orthogonal transforming; 
(d-2) quantizing data under gone the 
orthogonal transform; and 
(d-3) estimating a motion based on the 
supplied data and the quantized data. 

33- The method of coding multimedia information as 
defined in Claim 32 wherein in said step (d), each 
region in the frame where the source appears is 
coded adaptiveiy to a data-type of a first frame 
where the source has appeared. 

34. The method of coding multimedia information as 
defined in Claim 32 wherein when a first frame 
where the source has appeared is a frame different 
from an intra-frame, a region where the source 
appears in the frame is coded as an intra-block. 

35. The method of coding multimedia information as 
defined in Claim 32 wherein in said step (d) and in 
a block included in a region where a source free 
from time-varying-video appears, coded data of a 
block in a frame corresponding to and following the 
first frame where the source has appeared is a 
same as that of a corresponding block of a previous 
frame. 

36. The method of coding multimedia information as 
defined in Claim 32 wherein in said step (d-2) a 
quantization matrix is changed at every region 
where a source appears before quantizing the data. 

37. The method of coding multimedia information as 
defined in Claim 32 where in said step (d-2) based 
on analyzed scenario information, allotting data- 
types according to a ratio among region areas 
where respective sources appear, and changing a 
quantization matrix before quantizing the data. 

38. The method of coding multimedia information as 
defined in Claim 32 wherein in said step (d), storing 
data coded of a region where a source appears in a 
frame which has been coded as a first intra-frame 
since the source free from time-varying-video has 
appeared, and copying the stored data onto a 
region where the source appears in an intra-frame 
among following frames where the source is dis- 
played. 

39. The method of coding multimedia information as 
defined in Claim 34 wherein in said step (d), storing 
data coded of a region firstly as an intra-block— the 
region where a source free from time-varying-video 
appears —and, copying the stored data onto a a 
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region where the source appears in an intra-frame 
among following frames where the source is dis- 
played. 

40. The method of coding multimedia information as 
defined in Claim 35 wherein in said step (d) when 
data having a different data-type appears in a 
macro-block, and when information of "no change 
from previous data" can be inserted in each DCT 
block, DCT and quantization are omitted by insert- 
ing information which describes a coding pattern. 

41. The method of coding multimedia information as 
defined in Claim 32 wherein in said step (d-3) the 
search area defined is limited to a necessary area. 

42. The method of coding multimedia information as 
defined in Claim 32 wherein in said step (d), (i) 
when a first frame where video data of a still graph- 
ics appear is an intra-frame, a macro-block- 
included in a region where the video data is dis- 
played in a coded frame— is coded as an intra- 
block, where the coded frame follows the intra- 
frame and includes a predictive coded frame and a 
bi-directional predictive frame, and (ii) when a first 
frame where video data of still graphics appear is a 
frame other than an intra-frame. a macro-block 
included in a region where the video data is dis- 
played in the frame and a following frame is coded 
as an intra-frame, 

wherein in a region where the video data coded as 
the intra-block appears, quantizing the data by 
using a quantization matrix for an intra adaptively to 
the video data, and 

wherein in the predictive frame and the bi-direc- 
tional predictive frame, quantizing regions other 
than the region where the video data appears by 
using a quantization matrix for differential motion 
vector. 

43. The method of coding multimedia information as 
defined in Claim 32 wherein in said step (d), (i) 
when a first frame where video data of a still graph- 
ics appear is an intra-frame, in a frame other than a 
first intra-frame in due course of coding order fol- 
lowing the intra-frame, a macro-block included in a 
frame where the video data is displayed is coded as 
an intra-block, and (ii) when a first frame where 
video data of still graphics appear is a frame other 
than the intra-frame. the macro-block in the region 
where the video data is displayed in the frame is 
coded as the intra-block, 

wherein in a region where the video data coded as 
the intra-block appears, quantizing the data by 
using a quantization matrix for an intra-picture 
adaptively to the video data, 
wherein inserting escape information which indi- 
cates displaying a same video as a previous frame 



into a block other than the block coded as the intra- 
block of the region where the video data appears, 
and 

wherein in the predictive frame and the bi<Jirec- 
5 tionai predictive frame, quantizing regions other 

than the region where the video data appears by 
using a quantization matrix for differential motion 
vector. 

w 44. A recording media where data created through one 
of the methods of cocfing multimedia information as 
defined in Claim 29 through Claim 43. 
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